Efficient data loading in a data-parallel processor

ABSTRACT

A method of loading data into register files that correspond to respective execution units within a data-parallel processor. After receiving a first set of parameters that specify a subset of data within a first memory, the first set of parameters are compared to a plurality of sets of conditions that correspond to respective patterns of data. The first set of parameters is then converted to a second set of parameters in accordance with one of the sets of conditions satisfied by the first set of parameters. A sequence of memory addresses are generated based on the second set of parameters. Data is retrieved from locations within the first memory specified by the sequence of memory addresses and loaded into register files that correspond to respective execution units within a processor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from, and hereby incorporates byreference, U.S. Provisional Application No. 60/849,884, filed Oct. 6,2006 and entitled “System and Method for Efficient Software Developmentfor a Stream Processor.”

TECHNICAL FIELD

The disclosure herein relates to data processing systems generally andmore specifically to data placement in such systems.

BACKGROUND

In conventional program development systems, a human operator (aprogrammer) specifies keyword definitions, variable declarations andprogram functions through syntactical data entry into a text file,commonly referred to as a source file. The source file is compiled intoa sequence of machine-executable instructions through execution of acompiler (which is itself a computer program), and stored in an objectfile. The object file may be linked to one or more other object filesthrough execution of a linking program (e.g., a program which resolvesinter-object (as opposed to intra-object) references to functions,variables, definitions and so forth), resulting in creation of anexecutable code sequence stored as yet another file, called anexecutable file. In a general purpose data processing system, anoperating system (another program execution), responds to aprogram-execution command by retrieving a specified executable file froma relatively slow, non-volatile storage and placing the machine codefrom the executable file into a smaller, faster memory commonly referredto as main memory or system memory, and allocating storage for programvariables in the main memory. Thereafter, program execution occurs byrepeatedly fetching and executing instructions; fetching (retrieving)program instructions from main memory, loading the instructions into aninstruction register of a processor, and initiating instructionexecution in the processor.

FIG. 1A illustrates the actions of the programmer (100), compiler (102)and then hardware (104) with regard to conventional variable declarationand run-time reference. That is, a programmer initially declares thevariable through specification of a data type and a variable name asshown at 112. Thereafter, the programmer may specify an operation to beundertaken with respect to the variable by referencing the variable namein a program statement (e.g., the increment operation shown at 114).

Still referring to FIG. 1A, the compiler responds to the variabledeclaration by allocating an amount of storage space indicated by thedata type specifier, and by correspondingly extending the total datastorage space to be allocated to the executable program. The compilerconverts the variable reference (e.g., in the increment operation) intoa machine level load and/or store instruction that is included withinthe overall executable code sequence loaded from non-volatile storageinto a particular region of operating memory (i.e., placed in theoperating memory) by the operating system. In an embedded system or forelemental or kernel programs (e.g., basic input/output services or thelike), the executable code may be placed into a particular region ofoperating memory by a bootstrap loader (primitive program that copiesthe executable code to a predetermined location in the operating memory)or by more permanent disposition in a non-volatile memory (e.g., a readonly memory or any variant thereof).

During program execution, the processor executes the load/storeinstruction, resulting in issuance of a memory read request to anoff-chip memory subsystem. If the system includes an on-chip or off-chipcache, the cache will be queried (e.g., by comparing a portion of thememory address issued in the memory access request with contents of atag memory) to determine whether the data sought has been cached as partof a preceding memory access. If a cache hit occurs (data is in cache),the data will be retrieved from the cache and the off-chip memory accessrequest canceled. Otherwise, a cache miss occurs, and the off-chipmemory access request is completed to fetch the requested data to theprocessor. FIG. 1B illustrates the transfer of data from off-chip memory159 (random access memory (RAM) such as dynamic RAM (DRAM) or static RAM(SRAM)) to a processor 150 and cache memory 155 that are disposed on anintegrated circuit die 140. After the content of memory region ‘x’ isrecorded in the cache (i.e., as x′), a subsequent memory access directedto memory region ‘x’ will result in a cache hit, obviating off-chipmemory access.

Although the combined actions of the hardware and compiler serve to hidethe underlying complexity of memory access from the programmer, enablingthe programmer to simply declare variables without concern for theirplacement in the memory hierarchy represented by the off-chip memory andcache, the on-chip cache tends to be relatively large and slow and thuscompromises performance. Worse, in the event of a cache miss, which mayoccur whenever data has not yet been cached or has been evicted from thecache due to non-access or other reason, a substantial performancepenalty occurs in forcing access to off-chip memory.

In high-performance processing systems where the penalties associatedwith cache operation/cache miss are generally not tolerable, theprogrammer may specifically place critical data in an on-chip memorythat is immediately accessible to the processor. The immediate access tothe on-chip memory results in substantially faster access thanconventional on-chip cache architectures, and dramatically faster accessthan in instances of cache miss.

FIG. 2A illustrates the typical operations of a programmer 200, compiler202 and hardware 204 in a system in which the programmer specifies theon-chip address of critical data. Specifically, the programmer specifiesan on-chip address (e.g., as in the definition statement at 210) andanchors a variable at that address through declaration of a pointer to aspecified data type, and assignment of the on-chip address to thepointer as shown at 212. Thereafter, the programmer may specify areference to the on-chip address (i.e., access the content of thevariable anchored at the on-chip address) by dereferencing the pointer.This is shown at 214 by an exemplary C programming language statement inwhich the ‘*’ symbol indicates that the content at the address specifiedby pointer_variable_name (i.e., 0x60000) is to be incremented.

Still referring to FIG. 2A, a compiler converts the reference to theon-chip address (i.e., the dereferenced pointer) into machine-levelinstruction to load data from (and potentially to subsequently storeincremented at) the on-chip address. As before, the machine levelinstruction is fetched and executed by a processor, but in this case,due to the specification of on-chip address, execution results in directaccess to on-chip memory.

Although substantially higher run-time performance may be achievedthrough programmer specification of on-chip addresses, programdevelopment becomes substantially more complex, particularly whereprogram development is carried out by a team of individuals, as caremust be taken to avoid data placement errors (i.e., inadvertent overlapbetween on-chip storage space allocated to program variables as showngraphically in FIG. 2B by off-chip memory 159 and an integrated circuit240 that includes a processor 250 and on-chip memory 255). Arrangingdata in on-chip memory efficiently tends to be time consuming,lengthening the code writing process. Program debugging also tends tobecome more difficult as data placement errors are often hard to trace.In the worst case, depending on the nature of the data stored and testvectors exercised, the erroneous placement may not be detected at all,leading to release of defective software.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure herein is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1A illustrates actions of a programmer, compiler and hardware withregard to conventional variable declaration and run-time reference;

FIG. 1B illustrates a processor, cache and off-chip memory in which aprogram developed in accordance with operations shown in FIG. 1A may beexecuted;

FIG. 2A illustrates actions of a programmer, compiler and hardware withregard to variable declaration and run-time reference in ahigh-performance processing environment;

FIG. 2B illustrates a processor, on-chip memory and off-chip memory inwhich a program developed in accordance with operations shown in FIG. 2Amay be executed;

FIG. 3A illustrates actions of a programmer, compiler and hardware withregard to variable declaration and run-time reference in accordance withan embodiment;

FIG. 3B illustrates a processor, on-chip memory and off-chip memory inwhich a program developed in accordance with operations shown in FIG. 3Amay be executed;

FIG. 4 illustrates an embodiment of a simple processor;

FIG. 5 illustrates an embodiment of a processor with multiple registerfiles;

FIG. 6 illustrates another embodiment of a processor with multipleregister files;

FIG. 7 illustrates a state diagram for communication schedulingaccording to an embodiment;

FIG. 8 illustrates a graph for communication scheduling according to anembodiment;

FIG. 9 illustrates a graph for communication scheduling according toanother embodiment;

FIG. 10 illustrates a graph for communication scheduling according toyet another embodiment;

FIG. 11A illustrates a graph for communication scheduling according toan alternative embodiment;

FIG. 11B illustrates yet another embodiment of a processor with multipleregister files;

FIG. 11C illustrates a graph for communication scheduling according tostill another embodiment;

FIG. 12A illustrates an example set of input data and characteristicsthat may be used to define patterns of parallel data;

FIG. 12B illustrates an allocation of data retrieved from an inputbuffer;

FIG. 13 illustrates a generalized sequence of operations that may beperformed to retrieve data stored at arbitrary locations within an inputbuffer;

FIGS. 14A-14C illustrate data descriptors to characterize data to beretrieved from an input data buffer;

FIG. 15 illustrates a table of example data patterns;

FIG. 16 illustrates a pseudocode description of an example set of dataretrieval;

FIG. 17 illustrates a flow diagram for programmatically determiningparameters;

FIG. 18 illustrates a memory access engine according to an embodiment;

FIG. 19 illustrates a pseudocode description of an operation of a finitestate machine, according to an embodiment;

FIGS. 20A-20B illustrate example register file load operations accordingto various embodiments;

FIGS. 21A-21J illustrate parameter translations according to variousembodiments;

FIGS. 22A-22E illustrate visualizations for an interactive developmentenvironment, according to various embodiments;

FIG. 23 illustrates a communication between a host processor and aco-processor according to an embodiment;

FIG. 24 illustrates a communication between a host processor and aco-processor according to another embodiment;

FIG. 25 illustrates a block diagram that depicts a computer system uponwhich an embodiment may be implemented.

DETAILED DESCRIPTION

In embodiments of the invention disclosed herein, a programmer ispermitted to categorically control placement of data in either on-chipor off-chip storage by including or omitting an on-chip qualifier orkeyword. A compiler, when executed by a data processing system, respondsto the on_chip qualifier by determining an available location withinon-chip memory in view of previous allocations (resulting from responseto prior instances of the on_chip qualifier with respect to other datadeclarations) and allocating the determined location to theprogrammer-specified variable. By this arrangement, the programmer mayconveniently control whether data is placed on-chip or off-chip and thusavoid the performance penalty associated with the uncontrolled-placementapproach described in reference to FIGS. 1A and 1B. At the same time,the programmer need not specify the precise on-chip address of on-chipdata, and thus may avoid the inefficiency and susceptibility toplacement errors that plagues the precise-placement approach describedin reference to FIGS. 2A and 2B. Further, because on-chip data valuesmay be declared by data type and variable name, as opposed to beinganchored by a precise pointer assignment, on-chip data variables may bereferenced by the programmer by mere reference to the variable namerather than the more confusing pointer dereference described inreference to FIG. 2. From a program execution standpoint, theperformance benefits achieved through on-chip data placement are fullyrealized as the processor will execute the same machine/level load storeinstruction, but reference a compiler determined address instead of aprogrammer specified address.

In another embodiment of the invention, the compiler formulates acommunication scheduling as an instance of the well known“shortest-path” problem. This formulation incorporates adding passoperations and spill operations to avoid register file overflows and canbe solved optimally in sub-quadratic time. In one embodiment, thecompiler uses a conservative approximation to track the number of opencommunications for which routes can eventually be found. Based on theapproximation, it does not schedule routes which would make itimpossible to find a route for every open communication.

In various other embodiments of the invention disclosed herein, datacharacteristics are evaluated to generate a set of data descriptors thatmay be applied, during program compilation or system run-time, todetermine an efficient manner of retrieving arbitrarily located data andallocation of the retrieved data to individual execution lanes. The datadescriptors may be expressed by programmer-defined constants, run-timevariables, or a combination of constants and variables. In oneembodiment, program code having constant data descriptors definedtherein may be used to identify, at program compilation time and basedon the data descriptors, a pattern of data within an expected input datastream. If constant definitions are not sufficient to enable datapattern identification, data patterns may be identified during programexecution (i.e., during system run-time) based on variable datadescriptors, or a combination of variable and constant data descriptors.In any case, data descriptors may be iteratively evaluated to identifypossible data patterns and corresponding data transformations (i.e.,data retrieval sequences, also referred to herein as transforms),testing in progression for data patterns that yield more efficienttransformations to data patterns that yield less efficienttransformations, defaulting to a lowest-efficiency having assuredapplicability if no higher-efficiency transformations apply.

Compiler-Controlled On-Chip Data Placement

FIG. 3A illustrates exemplary operations of a programmer 300, compiler302 and hardware 304 in a system in which the programmer is enabled tocategorically, but non-specifically (i.e., without entering or otherwiseexpressing a precise on-chip address) specify placement of data ason-chip or off-chip, an approach referred to herein askeyword-controlled data placement. In the embodiment shown, theprogrammer may specify that a given data value (having a size andstructure according to the data type) is to be placed on_chip (i.e.,having a region of on-chip memory allocated for storage of the valuesassigned to the variable) by providing (i.e., typing into a source file)the qualifier or keyword “on_chip” as a prefix to the data typespecification, as shown at 312. Off-chip placement may be specified byomitting the on-chip keyword as in the declaration shown at 314.

Referring to FIGS. 3A and 3B, the compiler responds to the on_chipqualifier by determining an available location within on-chip memory 255(i.e., memory disposed on same integrated circuit die 240 as processor250) and allocating that location to the programmer-specified variable.Note that the location may encompass a range of memory addresses, aswhere the data type of the declared variable is an array, datastructure, double-word, floating point value or otherwise includesmultiple constituent bytes. When the on_chip qualifier is omitted from avariable declaration, the compiler allocates memory in an off-chipstorage (e.g., off-chip memory 159 shown in FIG. 3B).

The compiler converts a reference to a declared variable into amachine-level instruction to access (load and/or store) data at theallocated address, thereby accessing on-chip memory directly toload/store data that has been placed on-chip (i.e., variable declaredwith on_chip keyword), and accessing off-chip memory (or possiblyon-chip or off-chip cache) for data placed in off-chip memory (i.e.,declared without on_chip keyword and with an off-chip keyword). Theprocessor executes the load/store instruction by directly accessingon-chip memory at the compiler-determined address for variables declaredwith the on_chip qualifier and issuing request to access off-chip memory(and, if cache present, instruction/signals to query the cache) forvariables declared with the off_chip qualifier.

In cases where a loading agent (e.g., an operating system or otherexecuted program) loads program code into system memory and allocatesdata space for program execution (i.e., as opposed to predetermined codeplacement and data space allocation that may occur, for example, in anembedded application), the compiler may qualify on-chip addressreferences as being absolute rather than relative to ensure that data iscorrectly placed within on-chip storage. Other, off-chip locations maybe specified by the compiler as being relative to the starting addressof the data space for the program, an address that may not be resolveduntil program run time.

Still referring to FIG. 3, it should be noted that the on-chip qualifiermay be expressed by a keyword other than on_chip. Also, instead ofdefaulting to off-chip allocation in the absence of the on-chipqualifier, an explicit off-chip keyword (e.g., “off_chip”) may be usedto express off-chip allocation, and the compiler itself then mayautomatically determine placement of a variable declared without anon-chip or off-chip qualifier (referred to herein as an non-categorizedor unanchored variable). For example, in one embodiment, the compilermay determine the available on-chip memory based on explicitallocations, and then choose to locate frequently accessed or otherwisecritical data (as determined by the number of variable references thatappear in the compiled program code and/or source file or by a qualifierthat hints at data criticality) on-chip if sufficient on-chip storage ispresent. In another embodiment, a hierarchy of data placement qualifiersmay be provided to enable the programmer to specify, by providing afirst qualifier, that data is to be placed certainly on-chip, and then,by a second, different qualifier, that data is optionally to be placedon-chip at the compiler's option and based on various factors (e.g.,whether sufficient on-chip space exists and/or whether a thresholdnumber of references or frequency of references in proportion to totalnumber of variable references, etc. appears in compiled code or sourcecode). Also, where the target hardware platform includes multipledifferent classes of on-chip storage, multiple on-chip qualifiers, eachcorresponding to the different storage class may be available to theprogrammer to allow categorical control over data placement within thedifferent classes of on-chip storage without requiring specification ofparticular storage addresses. In such an embodiment, the compiler mayrespond to each different on-chip qualifier by determining a specificstorage location within the specified on-chip storage and allocatingsame to the declared variable.

According to one embodiment, a programmer or developer specifies thatdata buffers are to be placed in on-chip memory but the data buffers areactually placed by a compiler. With automated buffer placement, theprogrammer can simply specify a size of the buffer(s). For example, onepossible declaration syntax is:

stream type name (size);

The size of the buffer(s) can be specified using one of any number ofmethods. Three examples are:

-   -   1. Size is specified as a compile-time constant. The compiler        can statically allocate all buffers, or produce a compile time        error message if the required buffers do not fit in the on-chip        memory at any point in time.    -   2. Size is specified as a run-time variable. The compiler can        insert calls to a run-time allocation system to allocate and        de-allocate each buffer, or produce an error message if the        buffer does not fit.    -   3. A hybrid approach in which a maximum size is specified as a        compile-time constant, and the actual size is determined at        run-time subject to the stated maximum. This provides the        benefits of a compiler-time constant size with increased        flexibility.

Thus, when a compiler is placing a buffer (e.g., a buffer that isspecified by the developer to be on-chip), the compiler may produce anerror message when the set of such buffers cannot fit in the on-chipmemory. The following is example code that can be used to implement thecompile-time constant method described above.

// declare on-chip memory buffers, placed by compiler

stream int in_even_rows(SIZE / 2);

stream int in_odd_rows(SIZE / 2);

stream int out(SIZE / 4);

If an approach involving a compile-time constant size or maximum ischosen, the system can provide a mechanism for assigning differentvalues to one or more symbolic constants used in the constantexpressions for all buffer sizes in order to automatically determinesizes such that all buffers fit in on-chip memory. For example, if anentire image will not fit in memory, the code could be restructured todivide the image into strips and process each strip separately. A scriptor compiler could then be used to vary a constant to determine the stripsize:

stream int in_even_rows(WIDTH * _(——————)STRIP_NUM_ROWS);

stream int in_odd_rows(WIDTH * _(——————)STRIP_NUM_ROWS);

stream int out(WIDTH * _(——————)STRIP_NUM_ROWS / 2);

for (int strip=0; strip <HEIGHT / _(——————)STRIP_NUM_ROWS; strip++) {

-   -   // process each strip of the image    -   . . .

}

A script of the compiler could then try different values of_(——————)STRIP_NUM_ROWS to find the largest possible value. Thus, when acompiler is placing a buffer (e.g., a buffer that is specified by thedeveloper to be on-chip), the compiler may automatically size thebuffer(s) using a script or compiler to try different sizes by varyingsymbolic constant(s) and respond to over-allocation with smaller sizes.

Associating Buffers with an Access Pattern so that the Compiler canInsert Loads and Stores as Needed

The contents of the buffers can be managed using one of any number ofmethods. Three examples are:

-   -   1. Data can be loaded to or stored from each buffer explicitly        by the programmer.    -   2. A specific data access pattern, as described below under the        heading Efficient Data Loading In a Data-Parallel Processor, can        be bound to each buffer, and the compiler can load and store the        contents as needed.    -   3. A hybrid approach in which the programmer explicitly        specifies loads and stores but the compiler may add, remove, or        alter load and stores for optimization purposes or to enable        more buffers to fit in the on-chip memory.        The code from the example may be expressed using method one as        follows:

// declare on-chip memory buffers, placed by compiler stream intin_even_rows(SIZE / 2); stream int in_odd_rows(SIZE / 2); stream intout(SIZE / 4); // load input data spi_load(in_even_rows,&in_image[0][0], 0, image_height / 2, image_width, image_width * 2, 2,0); spi_load(in_odd_rows, &in_image[0][1], 0, image_height / 2,image_width, image_width * 2, 2, 0); // compute results for (int i = 0;i < size; i += 2) ( out[i / 2] = (in_even_rows[i] + in_even_rows[i +1] + in_odd_rows[i] + in_odd_rows[i + 1]) / 4; ) // store output dataspi_store(out, &out_image[0][0], size / 4, 1, 1, 1, 0);Using method two, the example may be expressed as:

// declare on-chip memory buffers, placed by compiler stream intin_even_rows(SIZE / 2, &in_image[0][0], 0, image_height / 2,image_width, image_width * 2, 2, 0); stream int in_odd_rows(SIZE / 2,&in_image[0][1], 0, image_height / 2, image_width, image_width * 2, 2,0); stream int out(SIZE / 4, &out_image[0][0], size / 4, 1, 1, 1, 0); //compute results, compiler inserts loads to fill and drain buffers asneeded for (int i = 0; i < size; i += 2) ( out[i / 2] =(in_even_rows[i] + in_even_rows[i + 1] + in_odd_rows[i] +in_odd_rows[i + 1]) / 4; )Using hybrid method three, the compiler would optimize the method onecode. For example, it might remove the store and a later re-load (notshown) if it was able to persist the out data in the on-chip memoryuntil it is used.

Data Transfer Scheduling for Multiple Register Files

A processor typically comprises a register file, which stores datavalues, and one or more arithmetic logic units (ALUs), which performoperations on selected data values. A compiler is a program executed bythe processor, which contains specific processes for routing data valuesbetween the register file and the ALUs. FIG. 4 illustrates an embodimentof a simple processor 100, comprising a register file 110 and ALUs 101and 102. On every clock cycle, each of the ALUs 101 and 102 reads inputdata values from the register file, performs a given operation on thedata values, then writes an output back to the register file 110. Sincethere is only a single register file 110, the compiler does not need todetermine where output data values are to be stored. In other words, anydata value output from either ALU 101 or 102 is simply re-inserted backinto the register file 110. However, the size of a register file growsexponentially in proportion to the number of values that can be readfrom or written to it each clock cycle. Thus, it may be desirable toreduce the number of values that can be read from or written to aregister file in a given clock cycle.

FIG. 5 illustrates an embodiment of a simple processor 200 with multipleregister files. The processor 200 comprises register files 210 a and 210b coupled to respective ALUs 201 and 202. In the embodiment of FIG. 5,each ALU 201 and 202 reads inputs from a respective register file 210 aand 210 b, and performs an operation on them. However, the processor 200is much more efficient than the processor 100, of FIG. 4, because asingle ALU may be allowed to write a value to a given register file (210a or 210 b) each clock cycle. Therefore, the register files 210 a and210 b may be much smaller than the register file 110, of FIG. 4.

When scheduling operations for a processor with multiple register files,the compiler generally needs to schedule hardware resources to transferthe outputs of each ALU to specific register files and, eventually, tothe inputs of other ALUs. The transfer of an output to an input is knownas a “communication” and the set of resources used to perform thecommunication is known as a “route.” Communication scheduling addressesthe problem of assigning each communication to a route. There are manyfactors which complicate communication scheduling.

As part of selecting a route, the compiler may need to insert “pass”operations to copy values between register files due to connectivitylimits, resource conflicts with other transfers, or to avoid overflowingregister files. Referring back to the example of FIG. 5, suppose anoutput of ALU 202 needs to be communicated to an input of ALU 201, butthe register file 210 a is completely full during the clock cycle inwhich ALU 202 needs to write its output. In order to avoid overflowingregister file 210 a, the data value output from ALU 202 may betemporarily stored back into register file 210 b. The compiler mustlater insert a pass operation to transfer the data from ALU 202 to theregister file 210 a. In the event that both register files are full, thecompiler may need to insert a “spill” operation to store the data toon-chip memory and reload it at a later time. However, spill operationsare very costly and inefficient, and should thus be used only insituations where no alternative is available. The issue of communicationscheduling may seem rather trivial, in the embodiment of FIG. 5, sincethere are only two register files (210 a and 210 b). However, the moreregister files (and ALUs) there are within a processor, the more complexcommunication scheduling becomes.

FIG. 6 illustrates another embodiment of a processor 300 with multipleregister files. The processor 300 comprises four register files 310 a,310 b, 320 a, and 320 b, coupled to two ALUs 301 and 302. In theembodiment of FIG. 6, ALU 301 is coupled to receive data values fromregister files 310 a and 310 b, and can write output data to any of theregister files 310 a, 310 b, 320 a, and 320 b, via Crossbar_A. On theother hand, ALU 302 is coupled to receive data values from registerfiles 320 a and 320 b, but can write output data only to register files310 b and 320 b, via Crossbar_B. Thus, in order to route an output fromALU 302 to register files 310 a or 320 a, one or more intermediate stepsmust always be scheduled. In other words, an output data value from ALU302 must first be routed through register file 310 b, and ALU 301, inorder to be written to register files 310 a or 320 a. Communicationscheduling for processor 300 is thus further compounded by taking intoaccount this necessary intermediate step.

A route can only be found for a communication once both the source anddestination operations have been scheduled, but the ability to find aroute must be preserved from the time the first operation is scheduledto the time the second operation is scheduled. Thus, suppose the sourceoperation is scheduled first. Until the destination operation isscheduled, the compiler must ensure that routes it chooses for othercommunications do not make it impossible to find a route due to resourceconflicts. A communication between a scheduled operation and anunscheduled operation is called an “open” communication. Referring backto the example of FIG. 6, suppose a first operation scheduled in ALU 302produces an output consumed by a second, as-yet-unscheduled, operation.Unless the compiler enforces the ability to schedule the communicationfrom ALU 302 to ALU 301, other pairs of communicating operations mightbe scheduled with communications that write to register files 310 b and320 b on the same clock cycle that ALU 302 needs to write its output.However, ALU 302 can only write to register files 310 b and 320 b, thusrendering the foregoing open communication impossible.

There are several alternative methods for performing communicationscheduling. The simplest is to assign all operations to specific ALUsand all inputs and outputs to specific register files before scheduling.Then, insert pass operations that move values between register files asneeded. Lastly, schedule all of the operations in time. Though simple,this method substantially restricts scheduling flexibility. For example,in reference to FIG. 6, if an operation is assigned to ALU 301 beforescheduling, it might be possible to schedule it on ALU 302 on an earliercycle during scheduling, but that option would be ignored by thecompiler. It is also possible to simply select a source or destinationregister file for an operation at random when it is scheduled, but thisleads to either restricted scheduling flexibility in scheduling theoperation(s) which with it communicates, or requires unnecessary passesto move values from arbitrary chosen register files to correct registerfiles.

More complex approaches run the risk of drastically increasing compiletime. For example, given multiple operations all trying to write outputson the same cycle, the compiler might enumerate the possible ways eachoutput can be written, then consider each possible permutation until oneis found in which all outputs can be written without resource conflicts.Unfortunately, the time required for the permutation search variesexponentially with the number of ALUs. Further, when trying to connectan output to an input using pass operations, the compiler might considerall possible placements of several pass operations between the twocommunicating operations. It would need to execute the exponential-costpermutation search for each pass operation placement (which is in turnexponential in the number of pass operations considered). Incombination, such challenges can result in compile times ranging fromminutes to days.

FIG. 7 illustrates a state diagram for communication schedulingaccording to an embodiment of the invention. The state diagram of FIG. 7is herein discussed in reference to the processor embodiment of FIG. 5.At the start and end of every clock cycle, a data value is in a statecorresponding to a particular register file (RF). During each clockcycle, a data value transitions between states along one of two types ofedges: “store” edges, wherein the data value is simply kept within thecurrent RF, and “pass” edges, wherein the data value is routed to adifferent RF. For example, a data value currently stored within RF 210 amay, over the next clock cycle, remain in RF 210 a (e.g. tracing thestore edge) or be routed, via ALU 201, to RF 210 b (e.g. tracing thepass edge). On the other hand, a data value currently stored within RF210 b may, over the next clock cycle, remain in RF 210 b or be routed,via ALU 202, to RF 210 a. In alternative embodiments, “spill” edges maybe included to indicate extreme circumstances where it may be necessaryto transfer a data value to on-chip memory, to be reloaded at a latertime. While state diagram of FIG. 7 is useful for locating a data valuewithin any given clock cycle, in reality, the compiler must trace thecomplete path of a communication over multiple clock cycles.

FIG. 8 illustrates a graph for communication scheduling according to anembodiment. The graph of FIG. 8 illustrates the states for each clockcycle in a given communication, wherein the path of a data value can betraced across the span of the entire communication. Pass and store edgesconnect the states in one clock cycle to the states in the next clockcycle. The “output” state corresponds to the start of the communication,or when a designated ALU outputs the data value. The “input” statecorresponds to the end of the communication, or when a designated ALUinputs the data value. In alternative embodiments, “spill” edges may beincluded to indicate extreme circumstances where it may be necessary totransfer a data value to on-chip memory, to be reloaded at a later time.For example, FIG. 11A illustrates a graph for communication schedulingaccording to an alternative embodiment, wherein spill edges (representedby dotted lines) connect the states in one clock cycle to on-chip memory1100 in the next clock cycle.

Any path along the graph of FIG. 8, from the output state to the inputstate, is a valid plan for when to store and when to pass the datavalue. For example, in reference to the processor embodiment of FIG. 8,suppose an output of ALU 201 needs to be communicated to an input of ALU202 after three clock cycles. According to the graph of FIG. 8, thereare four valid routes which address this communication scheduling.

In a first route, the data value is output by ALU 201 and stored in RF210 b during the first clock cycle. The data value remains stored in RF210 b during the subsequent second and third clock cycles, and is theninput to ALU 202.

In a second route, the data value is output by ALU 201 and stored in RF210 a during of the first clock cycle. The data value remains stored inRF 210 a for the duration of the second clock cycle. The data value isthen passed, via ALU 201, and stored in RF 210 b for the duration of thethird clock cycle, and is then input to ALU 202.

In a third route, the data value is output by ALU 201 and stored in RF210 a during the first clock cycle. The data value is then passed, viaALU 201, to be stored in RF 210 b for the duration of the second clockcycle. The data value remains in RF 210 b for the duration of the thirdclock cycle, and is then input to ALU 202.

In a fourth route, the data value is output by ALU 201 and stored in RF210 b during the first clock cycle. The data value is then passed, viaALU 202, to be stored in RF 210 a for the duration of the second clockcycle. The data value is then passed again, via ALU 201, to be stored inRF 210 b for the duration of the third clock cycle, and is then input toALU 202.

Given the above example, it is clear that the compiler must ultimatelychoose one “best” route for communication scheduling. And as moreregister files (RFs) are introduced, more possible permutations ofroutes arise, thus further complicating the task of choosing a bestroute. However, selecting a best route may vary each time, depending onthe given circumstances. In an embodiment, the compiler may consider allpossible permutations of routes for open communications. In alternativeembodiment, the compiler may choose a specific route for each opencommunication arbitrarily.

FIG. 9 illustrates a graph for communication scheduling according toanother embodiment of the invention. The graph of FIG. 9 is similar tothe graph of FIG. 8, with the exception of relative “weights”, denotedby the thickness of the edge, given to each of the (pass and store)edges. The assignment of weights to the edges may depend on manydifferent factors, depending on the “cost” or desirability of a givenroute (e.g. the greater the weight, the less desirable it is to tracethat edge). In an embodiment, pass edges are assigned a moderate weightif the corresponding ALU is not in use, or an infinite cost if it is.Performing a pass operations prevents the corresponding ALU fromperforming a useful operation, thus it may be desirable to avoid such asituation whenever possible. In another embodiment, store edges areassigned a weight based on how full the corresponding register file (RF)is, varying from zero (empty) to infinite (full). It is important tonote that tracing a store edge may not always be preferable to tracing apass edge. For example, a pass operation is always preferable to anyalternative that may cause a RF to overflow (e.g. by continuing to storea data value in that RF). In alternative embodiments, other factors orconsiderations, any combination of which may be included in determiningweight assignment. In yet other embodiments, spill edges may beincluded, having near-infinite weights associated with them. This isbecause, in most instances, a spill operation is only desirable when theonly alternative is to overflow a RF (i.e. no pass or store edge may betraced without overflowing a RF). In general, (lower) weights on theedges give preference to routes through the graph that avoid resourceconflicts, reuse resources for multiple communications of the sameoutput, minimize pass or spill operations, and do not occupy registersin near-full register files. In an embodiment, the weights are updatedduring the scheduling process to reflect the current schedule state.

An example is herein discussed in reference to the embodiments of FIGS.5 and 6. Suppose an output of ALU 201 needs to be communicated to aninput of ALU 202 after three clock cycles. Recalling the example above,with respect to FIG. 8, there were originally four valid routes forscheduling this communication. However, suppose the compiler knows thatRF 210 b will be heavily filled at the end of the first clock cycle (asindicated by the thickness of the edges connecting to RF 210 b of thesecond clock cycle). This information alone is sufficient foreliminating three out of four of the possible routes, thus isolating thebest route. For example, any routes tracing the data value to RF 210 bin the second clock cycle (e.g. the first and third routes of theprevious example) may be immediately eliminated, as such would cause RF210 b to overflow. Now only two routing options remain: wherein the datavalue is either initially stored in RF 210 a or in RF 210 b during thefirst clock cycle (e.g. the second and four routes, respectively, of theprevious example). If the data value is initially stored in RF 210 bduring the first clock cycle, it must then be passed, via ALU 202, to RF210 a where it is stored for the duration of the second clock cycle, andthen passed again, via ALU 201, to RF 210 b where it is stored for theduration of the third clock cycle, so that it can be subsequently inputto ALU 202. On the other hand, if the data value is initially stored inRF 210 a during the first clock cycle, it may remain in RF 210 a for theduration of the second clock cycle as well, before it is ultimatelypassed, via ALU 201, to RF 210 b where it is stored for the duration ofthe third clock cycle and then input to ALU 202. Note that former routerequires two pass operations whereas the latter requires only one.However, pass operations invariably consume resources (e.g. ALUs) whichare better served for other, more useful operations. Thus, it is oftenmore desirable to select the route requiring the fewest pass operations.Note, however, that depending on the weights assigned to other (store)edges, this may not always be the case.

By assigning weights to the edges of the graph, the compiler hasessentially broken down the problem of finding the best route to aninstance of the shortest-path problem. Thus, the compiler hassuccessfully narrowed down the four valid routes in this problem to asingle best route, as illustrated in FIG. 10 by a dotted line.

According to one embodiment, the compiler formulates the problem offinding a route for a communication as an instance of the shortest-pathproblem. This formulation incorporates adding pass operations and spilloperations to avoid register file overflows and can be solved optimallyin sub-quadratic time. One possible graph underlying the shortest pathproblem consists of one vertex per register file, for each cycle, plusadditional vertices and edges used to describe how values may propagatebetween register files. For example, a pass operation is modeled as aseries of edges and vertices that connects the vertex for a registerfile on cycle M with a vertex for a crossbar on cycle N=(M+passlatency−1) which is connected to a set of register file vertices oncycle N+1. Weights on the edges may give preference to routes throughthe graph that avoid resource conflicts, that reuse resources formultiple communications of the same output, that minimize pass or spilloperations, and that do not occupy registers in near-full registerfiles. Weights may be updated during the scheduling process to reflectthe current schedule state.

For example, consider scheduling a communication on multiple registerfile architecture 1108 illustrated in FIG. 11B. The architecture 1108contains two register files, register files 1110A and 1110B, and one ALU1112. The ALU 1112 supports a “pass” operation which will read a valuefrom either register file 1110A or register file 1110B and, with twocycles of latency, drive the value onto a crossbar 1114 connected to allregister files.

FIG. 11C shows one possible communication graph 1120 used to schedule acommunication on the multiple register file architecture 1108illustrated in FIG. 11B. For clarity, the communication graph 1120 isdivided into cycles. Each cycle contains two nodes for each registerfile, a “starts in” node labeled “Start A” or “Start B” and a “stays in”node labeled “Stay A” or “Stay B.” Each possible pass operation isrepresented by a series of nodes connecting a “starts in” node to thecrossbar node L cycles later, where L=(pass latency−1). For example, ifthe pass latency is 2 cycles, L=1.

The communication illustrated in FIGS. 11B and 11C is from an operationwhich writes its value onto the crossbar 1114 (depicted as 1122 in FIG.11 C) on cycle 4 to an operation which reads the value as an operandfrom the register file 1110A (depicted as 1124 in FIG. 11C) on cycle 8.Two possible paths through the graph 1120 are highlighted, one with asolid line and one with a dotted line. The solid line indicates a simplepath in which the value is written from the output bus to the registerfile 1110A (depicted as 1126 in FIG. 11C) and stays in the register file1110A until used. The dotted line indicates a more complicated path inwhich the value is initially written to the register file 1110B(depicted as 1128 in FIG. 11C) and then passed to the register file1110A (depicted as 1130 in FIG. 11C) later. Which path is chosen by thecompiler would depend on the weights assigned to the edges. For example,if the register file 1110A was very full, the edges used by the solidpath might be too costly. On the other hand, if the output bus wasalready being used at the end of cycle 7, then the dotted edge fromcrossbar 1114 on cycle 7 to Start A on cycle 8 (depicted as 1130 in FIG.11C) would have infinite weight, making the dotted path impossible.

In order to ensure that a route can always be found for every opencommunication, the compiler may use a conservative approximation totrack the number of open communications for which routes must eventuallybe found. The compiler then updates this approximation as it schedulesroutes for other communications. Any approximation should be capable ofensuring that each output can be written to a register file, each inputcan be read from a register file, and that pass operations can be usedto transfer the data value from the former register file to the laterregister file. However, different architectures may require differentapproximations.

In an embodiment, the approximation takes the form: M*x<=y, where M is aconstant matrix, x and y are vectors, and <= is true if and only if allthe elements in Mx are less than the corresponding elements in y. Eachelement of x represents a sum of the number of open communicationswriting to a given crossbar (ignoring duplicate communications of thesame data value). The value of y may vary, depending on which resourcesare in use during a given clock cycle, and can be stored in a lookuptable. The approximation should ensure that an ALU can write the outputof a source operation scheduled on it to any register file connected toa crossbar, which is also connected to a register file from which someALU could read the value and use it as an input of the destinationoperation. For example, in reference to the processor embodiment of FIG.9, on a clock cycle with no scheduled routes, the approximation requiresthe following to be true:

${\begin{pmatrix}1 & 0 \\1 & 1\end{pmatrix}*\begin{pmatrix}{x\; A} \\{x\; B}\end{pmatrix}} < \begin{pmatrix}2 \\4\end{pmatrix}$

Where xA corresponds to the number of open communications to registerfiles connected to Crossbar_A (i.e. register files 310 a, 310 b, 320 a,and 320 b), and xB corresponds to the number of open communications toregister files connected to Crossbar_B (i.e. register files 310 b and320 b). Thus, there can be at most two open communications to theregister files (310 b and 320 b) that both ALUs 301 and 302 can writeto, and at most four open communications in total. Accordingly, ifduring the same clock cycle, routes were scheduled that wrote to each ofthe register files except for register file 320 b, it follows that theapproximation would then require:

${\begin{pmatrix}1 & 0 \\1 & 1\end{pmatrix}*\begin{pmatrix}{x\; A} \\{x\; B}\end{pmatrix}} < \begin{pmatrix}1 \\1\end{pmatrix}$

Note that the vector [2 4] now becomes the vector [1 1], since a routemay be found for at most one more open communication.

Efficient Data Loading in a Data-Parallel Processor

Very-long instruction word (VLIW) processors and other data-parallelprocessors typically include multiple execution units or lanes thatexecute a shared sequence of instructions to process multiple streams ofinput data in parallel. VLIW system designers generally seek to maximizeefficiency by enabling as many execution lanes as possible to operate onvalid data in each execution cycle, a goal achieved primarily bymaintaining adequately loaded register files for each of the executionlanes. In many cases, data must be retrieved from scattered locations inmain memory and reorganized as it is transferred to the lane registerfiles. Unfortunately, the task of data retrieval and reorganization hashistorically been left to be carried out on a case-by-case basis byapplication programmers, often resulting in non-optimal allocation tolane register files and wasted execution cycles.

In various embodiments of the invention disclosed herein, datacharacteristics are evaluated to generate a set of data descriptors thatmay be applied, during program compilation or system run-time, todetermine an efficient manner of retrieving arbitrarily located data andallocation of the retrieved data to individual execution lanes. The datadescriptors may be expressed by programmer-defined constants, run-timevariables, or a combination of constants and variables. In oneembodiment, program code having constant data descriptors definedtherein may be used to identify, at program compilation time and basedon the data descriptors, a pattern of data within an expected input datastream. If constant definitions are not sufficient to enable datapattern identification, data patterns may be identified during programexecution (i.e., during system run-time) based on variable datadescriptors, or a combination of variable and constant data descriptors.In any case, data descriptors may be iteratively evaluated to identifypossible data patterns and corresponding data transformations (i.e.,data retrieval sequences, also referred to herein as transforms),testing in progression for data patterns that yield more efficienttransformations to data patterns that yield less efficienttransformations, defaulting to a lowest-efficiency having assuredapplicability if no higher-efficiency transformations apply.

Once a data pattern is identified and a corresponding datatransformation selected, data is retrieved from an input data buffer orother input data source according to the data transformation and loadedinto respective lane register files. In one embodiment, for example, thedata transformation is effected by generating a sequence of addressvalues that specify, within the larger buffer of input data, respectivedata substreams to be delivered to each execution lane. By thisarrangement, memory accesses may be carried out in the sequencespecified by the transformation to retrieve data for each execution laneand to store the retrieved data within the corresponding lane registerfile.

FIG. 12A illustrates an exemplary set of input data 1200 andcharacteristics that may be used to define patterns of parallel datatherein. In the particular example shown, the input data representpixels within a high-definition image (e.g., 1920 columns by 1080 rowsof 32-bit pixel values) and is occasionally referred to herein as aninput data stream as component values within the data may be receivedone after another or in groups in a serialized data transmission. Also,within the input data are shown, for purposes of illustration,collections of interesting data 1201 referred to herein as macroblocksor blocks, for short. Assuming, for example, that the input data 1200includes multiple columns of macroblocks 1201 to be processed within adata-parallel processor having a corresponding number execution lanes,and further that identical processing operations are to be carried outfor each of the macroblocks 1201, it becomes desirable to retrieve theconstituent data for each macroblock and load the retrieved data intothe lane register file for a corresponding execution lane. Morespecifically, as indicated by the designations “Lane 0 Data,” “Lane 1Data,” . . . , “Lane 15 Data,” the macroblocks in the leftmost columnmay be delivered to execution lane 0, with the macroblocks in columns1-15 similarly delivered to execution lanes 1-15, respectively.Unfortunately, the sequence of memory accesses required within the inputbuffer (also referred to herein as a main memory or source buffer) toretrieve a given macroblock is not, at least in this example, a linearlyprogressive sequence. Rather, each macroblock 1201 may be offset from anadjacent macroblock (or from a starting address within the inputbuffer), and may include distinct non-contiguous groups of contiguouslystored data values or records (note that the term ‘contiguous’ is usedherein to express storage locations addressed by consecutive addressvalues). Even more generally, each macroblock 1201 may be stored at anarbitrary (rather than systematic) location within the input buffer.Accordingly, in one embodiment, data descriptors that are used toparameterize or describe a set of input data include, but are notlimited to:

Descriptor Example in FIG. 12A Offset Offset between implicit orexplicit reference Offset between index and start point within inputbuffer and start of data of macroblock Count Number of non-contiguousmemory accesses Height of macroblock required to retrieve datasub-stream (number of Groups) Group Number of contiguous locations to beaccessed Width of macroblock for each count Stride Address step betweengroups within same data Row length (1920 in this sub-stream example)Index Explicit reference point for start of data sub- Indices for eachof stream macroblocks for lanes 0-15

In the context of input data 1200, for example, the ‘group’ parameterspecifies the width of a macroblock 1201 (pixel values in 16 adjacentcolumns), the ‘count’ parameter specifies the height of a macroblock (16pixel values in adjacent rows as shown in detail view 1215), the‘stride’ parameter specifies the step from group to group within a givenmacroblock (i.e., the row length of the input buffer, or 1920 in thisexample), and the ‘index’ parameter includes or points to one or moreindex values that establish reference points which, when summed with the‘offset’ parameter, specifies the start of respective macroblocks.

Another data descriptor, referred to herein as the transpose, indicatesthe number of consecutively retrieved data values to be loaded into eachlane register file before advancing to the next lane register file. FIG.12B, for example, illustrates allocation of data retrieved from theinput buffer of FIG. 12A when the transpose value is sixteen, the sameas the group value. More specifically, assuming that there are alsosixteen lane register files, LRF0-LRF15, the first group of sixteen datavalues retrieved from the input buffer (i.e., from addresses 0-15) arestored in LRF0, the second group of sixteen values retrieved from theinput buffer (i.e., from locations 16-31) are stored in LRF1, and soforth to the sixteenth group of sixteen data values (i.e., retrievedfrom buffer locations 240-255) which are stored in LRF15. Thereafter,the data retrieval and register-file load operations are repeated afterstriding to the next sixteen groups of data values (from bufferlocations 1920-1935, 1936-1951, . . . 2160-2175), and then the retrievaland load operations are repeated another fourteen times to load data forthe first sixteen macroblocks into the lane register files. By thisarrangement, each of sixteen execution lanes may be supplied with arespective one of sixteen macroblocks of data that span a portion of amacro-block row within the input buffer, without requiringapplication-level programming of the precise sequence of buffer readsand register-file loads. After each set of sixteen macroblocks has beenprocessed by the execution lanes (or whenever the lane register fileshave been sufficiently drained to provide room for loading or startingto load the next set of macroblocks), the second set of sixteenmacroblocks may be retrieved from the input buffer (advancing toleft-to-right to the next column of sixteen macroblocks within the samerow, or advancing top to bottom to the next row of sixteen macroblockswithin the same column.

FIG. 13 illustrates a generalized sequence of operations that may beperformed to retrieve data stored at arbitrary locations within an inputbuffer and load the retrieved data into the lane register files of aVLIW or other data-parallel processor. Initially, the data ischaracterized according to a pattern-describing subset of the datadescriptors (i.e., pattern descriptors), including, for example, theoffset, count, group, stride and index parameters described above. Thedata characterization may take place at program development time, forexample, by recording constant values for offset, count, group, strideand/or index. Alternatively, data may be characterized at programrun-time, for example, by evaluating the data to discern trends or otherindications of data format or organization. Also, characterizations ofthe data may be received from an external source, for example, in theform of packet header information or other information receivedseparately from the data itself, but that indicates data format ororganization. However determined, run-time data characterization may berecorded in variable pattern descriptors that are applied in absence of(or instead of or as supplements to) statically defined patterndescriptors.

Whether statically or dynamically determined (or some combinationthereof), the pattern descriptors are applied at 1303 in conjunctionwith another subset of the data descriptors (or a single datadescriptor) that defines an allocation of retrieved data to individualexecution lanes (i.e., one or more lane allocation descriptors, such asthe transpose parameter discussed above). For example, lane allocationdescriptors may be applied in conjunction with constraints of theunderlying hardware set to determine which of multiple applicable datapatterns (determined by the pattern descriptors) will result in the mostefficient delivery of data substreams to the individual execution lanes.Thereafter, at 1305, a sequence of input-buffer addresses is generatedaccording to the identified pattern, to specify a collection of datareferred to herein as the transformed data. At 1307, the transformeddata is loaded directly into the lane register files or into a temporarybuffer before being loaded into the lane register files, with the databeing distributed among the lane register files in either case accordingto lane allocation descriptor at 1307.

Still referring to FIG. 13, an exemplary data transformation operationthat corresponds to the general transformation operation at 1305 isshown at 1309. In the particular embodiment shown, a function “spi_load”is invoked along with parameters that include a source buffer pointer,‘srcb’ (e.g., pointer to an input buffer 1311), pattern descriptors(offset, count, group, stride and indices) and a lane allocationdescriptor (transpose), and executed to generate a data transformation1312 that may be applied to retrieve parameter-specified data values (orrecords) from the source buffer. In the particular example shown, theoffset parameter is zero, the count is 48, the group is one, and thestride is four, meaning that a data substream may be extracted from theinput buffer by retrieving 48 groups of one data value each, with astride of four from group to group. The index parameter is null, meaningthat there are no indices and thus only one data substream to beretrieved for a given input data set (or at least per invocation of thespi_load function). Thus, referring to the data transformation 1312(i.e., the sequence of contiguous addresses from which data is retrievedto form the transformed data set), forty eight data values are retrievedfrom input buffer locations 0, 4, 8, . . . , 184, 188 (the finallocation being (count−1)*stride+(group−1), where ‘*’ denotesmultiplication) to form the transformed data set. An exemplary loadingof transformed data into a set of sixteen lane register files L0-L15(i.e., a specific case of the more general load operation at 1307) isillustrated at 1310. As shown, with a transpose of 3, each contiguousset of three data values specified by the data transform is loaded intoa respective and progressively higher numbered one of the lane registerfiles. That is, records from the first three source buffer locationsspecified by the data transform (i.e., source buffer locations 0, 4, 8)are loaded into L0, values from the next three source buffer locationsspecified by the data transform (i.e., locations 12, 16, 20) are loadedinto L1 and so forth to LR15 which receives values from the final threesource buffer locations specified by the data transform (i.e., locations180, 184, 188). As with other examples herein, the specific parameters,parameter values, number of lane register files and so forth areprovided for purposes of illustration only and may be varied inalternative embodiments.

FIGS. 14A-14C illustrate further uses of data descriptors tocharacterize data to be retrieved from an input data buffer (or inputdata stream). FIG. 14A, for example, shows the use of the offset, groupand count descriptors to specify a data substream stored at sixteencontiguous locations (i.e., locations 7-22) within an input buffer. Thatis, the offset is seven, the group size is one, and the count-(thenumber of groups) is sixteen, thus specifying input buffer storagelocations 7-22 as the source of the data substream. FIG. 14B providesanother example, in this case a data substream formed by two groups ofeight data values each (i.e., count=2, group=8). As shown, a stride often is used to advance from the start of one group to the start of thenext group within the data substream, but no offset (stride=10,offset=0). FIG. 14C illustrates another example of a data substreamformed by eight groups of four data values each (count=8, group=4), withno offset and a stride of eight between the start of adjacent groups(stride=8, offset=0). FIG. 14C further demonstrates the allocation ofthe data substream to a data-parallel processor having sixteen executionunits and sixteen corresponding lane register files (L0-L15). Morespecifically, a transpose of 2 is specified so that each four-recordgroup is split between adjacent register files. That is, records fromlocations 0-3 of the first group are split 0, 1 and 2, 3 between laneregister files L0 and L1, respectively; records from locations 8-11 ofthe second group are split 8, 9 and 10, 11 between lane register filesL2 and L3, respectively and so forth to lane register files L14 and L15,which receive records 56, 57 and 58, 59, respectively, of the eighthgroup.

FIG. 15 is a table of exemplary data patterns and their descriptorcharacteristics. When applied in conjunction with a given hardware set,the data patterns from 1 to 10 constitute a pattern hierarchy that yieldprogressively less efficient processing. Thus, in one embodiment, datadescriptors for an input data stream are iteratively compared with thecharacteristics (or conditions) defined for each data pattern in thetable (referred to herein as a pattern hierarchy table 1500), startingwith data pattern 1 and progressing toward data pattern 10 until a datapattern that fits the data descriptors is identified. To ensure thatdata of interest may be successfully loaded into lane register files inall instances, data pattern 10 is a degenerate case that corresponds topatternless data and so may be used with any arbitrary data arrangement.Accordingly, if the incoming-data descriptors fit none of the moreordered data patterns 1-9, data pattern 10 is applied as a fail-safe toensure that the data may be processed. By contrast, as the datadescriptors are iteratively tested against the conditions set forth forpatterns 1-9, the testing may be halted as soon as a match is found, andthe descriptor-matching data pattern applied to produce the datatransform.

The particular conditions set forth in the pattern-hierarchy table ofFIG. 15 are further described below in the context of particular datapatterns. It should be noted that the pattern-hierarchy conditions areprovided for example only, however, and generally result fromlimitations in the underlying hardware set. Consequently, differenthardware sets having different limitations may include differentconditions and also more or fewer conditions and/or numbers ofhierarchical data patterns.

Following the column order in the pattern hierarchy table 1500, the“Zero-valued transpose” condition is met when the transpose parameter iszero (patterns 7, 8 and 9 only), and is a special case indicating thatthe same data substream is to be loaded into each lane register file ofthe data-parallel processor. In one embodiment, the “Indexed data”condition is met whenever the index parameter is non-null. A non-nullindex parameter indicates that there are one or more indices (e.g.,recorded statically as a number of indices and their values, ordynamically in a linked list or other structure that allows a variablenumber of indices to be specified or determined) that correspond torespective data substreams. The “Integral groups per transpose”condition is met when the transpose parameter is an integer multiple ofthe groups parameter (i.e., transpose % group=0, where ‘%’ denotes amodulus operation). “Integral transposes per block” is met when thereare an integral number of transposes per block (i.e., (count*group) %transpose=0, where ‘*’ denotes multiplication), “Stride equal group” ismet when the stride and group are equal, “Transposes per group equal2^(n)” is met when there are 2^(n) transposes per group (i.e., 1, 2, 4,8, etc., up to some hardware supported limit), and, similarly,“Transposes per block equal 2^(n)” is met when(count*group)/transpose=2^(n).

FIG. 16 is a pseudocode listing of an exemplary set of data retrieval,transformation and register load operations carried out by a registerload function, “spi_load”. As discussed in further detail below, theactual hardware operations performed to effect the data transfer maydiffer from the generalized set of operations set forth in thepseudocode listing, but generally effect the same data transfer result.As shown, the data transfer starts with a FOR loop at line 110, loopingthrough each index in the linked list (or other data structure) pointedat by the ‘indices’ parameter. Within the FOR loop, at line 120, astarting location within the input buffer is established by adding thecurrent value of index (in the first iteration, the first index in thelist) to the ‘offset’ parameter. At line 130 a nested loop is begun, tobe repeated a number of times according to the ‘count’ parameter. Ineach pass through the nested loop, ‘group’ records are selected (line140) before advancing an input-buffer pointer (i.e., an address valuethat points to the next input-buffer location from which a record is tobe retrieved) by ‘stride’ records (150). The nested loop is concluded atline 160 and the outer FOR loop is concluded at line 170. Starting atline 200, the records selected in lines 110-170 are distributed to theexecution lanes. That is, while selected records remain (per the Whileloop at line 200), a nested For loop is begun at line 210 to load anumber of records according to the ‘transpose’ parameter into theregister file for each execution lane (line 220). The For loop isconcluded at line 230 and the While loop at line 240.

FIG. 17 is a flow diagram illustrating the manner in whichprogrammatically determined parameters (i.e., parameters determinedthrough software execution or defined statically as constants to beapplied during software execution) may be applied to generate hardwareparameters for carrying out the data transfer from a system memory orother input data source to lane register files. At 1751, one of Ndifferent data patterns is identified based on the programmaticallydetermined parameters (i.e., the software parameters). This operationmay be performed as described in reference to FIG. 15 by iterativelytesting the software parameters against sets of conditions to select adata pattern with which the software parameters comply and which yieldsmore efficient processing than other data patterns with which thesoftware parameters may comply. At 1753, the software parameters aretranslated into hardware parameters based on the identified pattern.That is, viewing the software parameters as a virtualization of theunderlying hardware set (i.e., corresponding to an idealized orgeneralized hardware set), the software parameters may include one ormore parameters that do not exist in the actual hardware, but that maybe translated into one or more hardware-supported parameters, referredto herein as hardware parameters, to control hardware operation. At1755, the hardware parameters are loaded into a state machine or othercircuitry for carrying out memory access operations, referred to hereinas a memory access engine (MAE). In one embodiment, the memory accessengine includes circuitry for generating a sequence of memory addressesin accordance with the data pattern parameters and controlling transferof data between each memory address in the sequence and the laneregister files in accordance with the lane allocation parameter(transpose). More specifically, the memory access engine includescircuitry to generate a sequence of addresses and register load signalsin accordance with the hardware parameters derived from counterpartsoftware versions of data pattern parameters and lane allocationparameters. At 1757, the memory access sequence is initiated within thememory access engine to transfer data from system memory into the laneregister files.

FIG. 18 illustrates an embodiment of a memory access engine 1875 thatincludes registers 1877 (or fields within a unified register) forstoring hardware parameters and a finite state 1879 machine forgenerating a sequence of input buffer addresses (IBAddr) andregister-file enable signals (le0-le4) in response to the hardwareparameters. In one embodiment, the registers 1877 include a recordlength register 1881, group register 1883, stride register 1885, offsetregister 1887 and set of index registers 1889. The record lengthregister 1881 provides storage for a hardware record length value(hw_reclen) that indicates the number of contiguously stored records(i.e., stored at sequential addresses in an input buffer or systemmemory) that are to be loaded into a given lane register file beforeadvancing to load the next lane register file. The group registerprovides storage for a hardware group value (hw_group) that indicatesthe number of lane register files to be loaded with hw_reclen recordsbefore either striding the input buffer address according to a hardwarestride value (hw_stride) or stepping to the next hardware index(hw_index) at the head of a queue formed by registers 1889. The offsetregister 1887 provides storage for an offset value (hw_offset) that isadded to each newly selected hardware index to generate an input bufferaddress. As shown, in one embodiment, the hardware record length,hardware stride and hardware offset registers (1881, 1885, 1887) areloaded with the software transpose, stride and offset parameters,respectively, and the hardware group register 1883 is loaded with avalue determined by dividing the software group parameter by thesoftware transpose parameter, an operation that may be performed withinadditional hardware circuitry dedicated to that purpose or a programmedprocessor.

The pseudocode listing at 1895 illustrates, in this example in the Cprogramming language, a manner of loading the hardware index registers1889 with values determined from the software indices. In the particularimplementation shown, an outer FOR loop is executed iterativelyaccording to the number of software indices (i.e., incrementing aninitially zero-valued loop variable, ‘i’, until the loop variable is nolonger less than the number of software indices). Within the outer FORloop, a temporary address variable, ‘c’, is assigned the i^(th) value ofthe software index (i.e., c=index[i]), and then an inner FOR loop isexecuted iteratively according to the number of groups per index (i.e.,incrementing an initially zero-valued loop variable ‘j’, until the loopvariable is no longer less than the software parameter, “count”). Withinthe inner FOR loop, the value of the temporary address variable isloaded into the index register indicated by a hardware index count value(initially zero to point to head-of-queue register hw_index0), and thenthe temporary address variable is incremented by the software strideparameter and the hardware index count incremented so that the softwareindex plus software stride is recorded in the next index registerlocation (e.g., hw_index1). By this arrangement, a pointer to eachsoftware-specified group is stored within a respective index register asa hardware index. In one embodiment, the hardware index registers areoperated as a shift register having a tail register (i.e., which holdsthe most recently loaded hardware index) and a head register (e.g.,hw_index0, which holds the least recently loaded hardware index) andlogic to shift the entries from one register to the next, advancing fromtail register to head register in response to an advance signal 1890(adv) from the finite state machine 1879. By this arrangement, thefinite state machine 1879 may step through a sequence of hardwareindices associated with a given data substream.

The finite state machine 1879 includes inputs to receive the hardwarerecord length, hardware group, hardware stride, hardware offset andhardware index values from registers 1877, and outputs to output aninput buffer address (IBAddr, which may be maintained as a statevariable within a register 1893 of the finite state machine 1879) andload-enable signals, le0-le4, to each of five lane register files (theremay be more or fewer lane register files and thus more or fewerload-enable signals in alternative embodiments), to enable retrieveddata to be stored therein. FIG. 19 is an exemplary pseudocodedescription of the operations of the finite state machine of FIG. 18 inresponse to a triggering signal 1892 (Enable), illustrating a manner oftransferring data from the system memory to the lane register files. Atline 110, loop variables i, j and k, which are used to specify thecurrent lane register file, offset relative to the current input bufferaddress, and hardware index, respectively, are initialized to zero. Atline 120, the hardware index count (the number of hardware indices asdetermined, for example, in the operations shown in FIG. 18 at 1895) isevaluated to determine if any hardware indices for the current datasubstream have been loaded into the hardware index registers. If thehardware index count is zero (no hardware indices), then the datasubstream consists of non-indexed data, and the operations shown inlines 130-230 are executed to complete the register file load operation.If there are hardware indices (i.e., hardware index count>0), the datasubstream is indexed, and the operations shown in lines 250-370 areexecuted to complete the register file load.

Turning first to the load operations for non-indexed data, at line 130,the input buffer address is assigned the value of the hardware offset.Then, at line 140, a loop is established to iteratively execute theoperations at lines 150-220 so long as the lane register file is notfull. More specifically, at line 150, a first nested loop is begun torepeat execution of lines 160-200 a number of times according to thehardware group parameter. At line 160 a second nested loop is begun torepeat execution of the operations at lines 170 and 180 according to thehardware record length. By this arrangement, the operations at lines 170and 180 are repeated hw_group*hw_reclen times (‘*’ denotingmultiplication), with a record (or data value) being retrieved from theinput buffer (or system memory or main memory) at the location specifiedby the input buffer address plus the relative offset value ‘j’ withinthe current hardware group (e.g., initially the hardware offset, thenthe hardware offset plus 1, 2, 3, . . . etc.) and loaded into the laneregister file specified by loop variable ‘i’. At line 180, the relativeoffset, ‘j’ is incremented to select the next record number. Whenhw_reclen number of records have been retrieved from the input bufferand loaded into the current lane register file, the second (inside)nested loop is completed at line 190, and the next lane register file isselected by incrementing the register file index, ‘i’ at 200.Thereafter, the next hw_reclen number of records are retrieved from theinput buffer loaded into the newly selected lane register file inanother repetition of the nested loop between lines 160 and 190. Thisoperation is repeated according to the number of hardware groups beforethe first nested loop (begun at line 150) is concluded at lines 210. Atline 220, the input buffer address is advanced by the hardware stridevalue, thus selecting the next set of memory locations within the inputbuffer from which data is to be retrieved and loaded into the next setof lane register files.

FIG. 20A illustrates an exemplary register file load operation effectedby the operations at lines 130-230 of FIG. 19 (i.e., loading non-indexeddata), including the relationship between software parameters 2011 andthe corresponding hardware parameters 2013 that may be loaded into thehardware registers of FIG. 18. In the particular example shown, thesoftware offset=2, group=4, count=8, stride=6 and transpose=2, so thathw_offset=2, hw_reclen=2, hw_group=2, and hw_stride=6. Consequently,data is loaded into the eight lane register files as shown at 2015.

Returning to FIG. 19, if the data is indexed, the operations at lines250-370 are performed, starting with an outer loop at line 250 that isrepeated so long as the lane register files are not full. A nested whileloop is begun at line 260 and repeated until all hardware indices withinthe hardware index register have been processed (i.e., until the loopvariable ‘k’ is no longer less than the number hardware indices). Atline 270, the input buffer address is assigned the value of the hardwareoffset plus the initial hardware index (hw_index[k]) before beginninganother loop at line 280, which provides for a number of iterations ofthe operations between lines 290 and 330, according to the hw_groupparameter. At line 290, another nested loop is begun to repeat theoperations at lines 300 and 310 a number of times according to thehw_reclen parameter. Together, the loops at lines 280 and 290 and theoperations at lines 300, 310 and 330 effect the lane register fileloading operations described in reference to lines 150-200, iterativelyloading hw_reclen records into each lane register file and advancing tothe next lane register file to do the same, until all the records withina hardware group have been loaded. At line 350, the hardware indexselector variable (i.e., loop variable ‘k’) is incremented to select thenext hardware index, and then the operations at lines 270-350 arerepeated. Overall, through repetition of the while loop at line 260 foreach hardware index in the hardware index register (i.e., ‘k’ times),the records pointed to by each hardware index (accounting for anyhardware offset though summation with hw_offset) are selected and loadedinto lane register files. FIG. 20B illustrates this operation, withexemplary software parameters 2021 that include offset=2, group=4,count=2, stride=6, and indices=0, 15, 30, etc. Following thesoftware-to-hardware parameter conversion (or translation) discussedabove yields hardware parameters 2023 that include hw_offset=2,hw_reclen=2, hw_group=2, and hw_indices=0, 6, 15, 21, 30. Note that thehardware indices (hw_indices) capture the software strides, and that nohardware stride is applied in this embodiment for indexed data. Loadingof lane register files in a hardware set having eight execution lanes isshown, for example, at 2025.

FIGS. 21A-21J illustrate parameter translations for each of the variouspattern types shown in the table of FIG. 15.

Multi-Level Visualization with Source Code Correspondence

Using existing interactive development environments IDEs, programmersoften need to add profiling code or use external tools to discoverperformance limitations. Writing profiling code or correlating theoutput of external tools with code in the IDE adds development overhead.Further, most existing IDEs are aimed at general purpose processors nothigh performance processors. For instance, existing IDEs may not displayvalues striped across SIMD lanes.

The development system (i.e., tool or set of tools for developingsoftware) introduces an interactive development environment (IDE) thatcontains several innovations:

-   -   1. It integrates visualization functions which help a programmer        understand program performance at multiple levels alongside the        conventional IDE functions such as editing and debugging.    -   2. It includes functions for debugging programs executing on        stream processors, which allow the programmer to see how the        program interacts with advanced hardware features such as SIMD        processor lanes and memory managed explicitly with DMA.        -   a. In particular, it provides an intuitive grid view of            vector variables with processor lanes along one axis and            variable names along another.

Specifically, the IDE may contain a subset or superset of the followingviews (figures provided to illustrate different views show highlysimplified instances of the view described):

-   -   Processor schedule view: view showing schedule of        processor-level operations, such as DMA transfers and        computation intensive code regions, e.g. kernels, loop-nests,        either projected from source code or based on an actual        execution trace. This view can be further enhanced by showing        information about inputs and outputs (such as streams or array        regions), dependences between operations, and scheduling        bottlenecks such as resource saturation or contention. See FIG.        22A for one possible example of this view.    -   ALU schedule view: view showing schedule of ALU-level        operations, such as additions, multiplies, and loads and stores,        either based on static compiler schedule or an execution trace.        This view can be further enhanced by showing information about        inputs and outputs (such as variable names, values, and/or        register locations), dependences between operations, and        performance bottlenecks such as the critical path of a single        block or the operations with dependences that form        strongly-connected component(s) in a loop. See FIG. 22B for one        possible example of this view.    -   On-chip memory contents view: view showing the contents on an        on-chip memory. One possible instance of this type of view shows        large memory blocks such as streams or array regions for the        entire on-chip address space or a region thereof, enabling a        macroscopic understanding of memory layout. Another possible        instance combines this view with processor-level schedule        information to show how the macroscopic layout changes over        time. See FIG. 22C for one possible example of this view.    -   Processor operation history/future: view showing list of        processor-level operations (as defined above), which have been        queued and/or executed by the relevant processor, either based        on actual run-time state or specific point in an application        trace. See FIG. 22D for one possible example of this view.    -   SIMD debugging view: view showing processor state or data        striped across SIMD processor lanes. One possible instance of        this type of view uses a grid with processor lanes along one        axis and variable names along the other axis. See FIG. 22E for        one possible example of this view.    -   Conventional IDE views showing source code, call stack, etc.

Any of these views may incorporate standard graphical functionality,including but not limited to, the ability to zoom in or out, scroll,enable or disable particular visual elements, search for and highlightspecific visual elements, and select visual element (s) to display moreinformation. Further, any of these views may display a subset orsuperset of the information described.

Each of these views is useful as a stand-alone application, and mayintroduce innovative aspects that improve the utility of such anapplication. However, by innovatively combining some or all of the aboveviews in the same IDE, it is possible to navigate quickly betweenrelated views showing different aspects of program performance orfunctionality, thereby improving programmer efficiency. The IDE maycontain some subset or superset of the following navigationalmechanisms:

-   -   Navigation from processor-level operations within the processor        schedule view to corresponding source code, or vice versa. For        example, navigation from a DMA operation in an execution trace        to lines that issue the DMA command within source code, or from        a kernel or loopnest to lines that define that kernel or        loopnest within source code.    -   Navigation from processor-level operation to corresponding ALU        schedule view(s) and vice versa.    -   Navigation from ALU-level operation(s) within ALU schedule view        to corresponding source code, or vice versa.    -   Navigation from on-chip memory contents view to corresponding        source code, or vice versa. For example, navigation from a        stream to the declaration of that stream in source code.    -   Navigation from on-chip memory contents view to corresponding        processor-level operations within processor schedule view. For        example, navigation from a stream or array region to the DMA        operation used to load or store it.    -   Navigation from pending processor operation view to        corresponding source code, or vice versa.    -   Navigation from SIMD debugging view to corresponding line(s)        within a source code view, or vice versa. For example, from a        variable to the declaration or all uses of the variable.

Navigation between views may be accomplished by one or more of a varietyof interface idioms, such a mouse clicks, pressing specific key(s), orselecting a command from a menu, or a combination of such idioms such asa right-button click followed by a menu selection.

An example may serve to illustrate the utility of the different views,and the utility of the ability to easily navigate between them in asingle integrated environment. For example, suppose a developer isattempting to optimize an application. The developer (e.g., a humanoperator) first looks at the program schedule view, and notices a kernelwhich dominates execution time. The developer then clicks on the kernelto bring up the kernel schedule view and sees an extremely long chain ofdependent operations which is limiting performance. The developer mayclick on an operation in the middle of the chain to bring up the sourcecode for that kernel, with the line containing the operation in questionhighlighted. The developer may toggle back and forth between the twoviews, clicking on different operations until the dependences areunderstood and a way to break the chain into parallel operations isdetermined. The developer makes the change, but executing the programreports that the output is now incorrect. The developer places abreakpoint at the end of the changed kernel, then inspects the on-chipmemory contents view to determine which kernel output is incorrect. Thedeveloper clicks on the output in the on-chip memory view and is takento the declaration in the source code, from which the code that computesthe output value is found. The developer single-steps through that codelooking at vector variables in the SIMD processor core view untilfinding the bug. Lastly, the developer uses the quantitative data fromthe processor operation history to compute the exact speedup deliveredby the optimization.

Hardware/Software Recording of Co-Processor Command Execution Data

A stream processor, or other high-performance processor, may beimplemented as co-processor specifically designed to executecomputation-intensive kernels within the context of a largerapplication. A host CPU executes the main application code in a system.It sends commands to the co-processor using a command queue as shown inFIG. 23.

It is important to be able to collect accurate performance informationabout the commands executed by the co-processor. Such information allowsthe software to perform analysis and visualization of applicationperformance. However, collecting this information using either softwareor hardware alone is unsatisfactory. Executing software on theco-processor to collect the data may be expensive enough to distort thedata being collected. Adding hardware with enough memory to record along performance trace consumes valuable chip area.

The development system includes an innovative mechanism for recordingco-processor performance data that combines a short hardware trace withsimple software for collecting multiple short trace segments andaggregating them into a larger trace. This hybrid mechanism provides thebest of both worlds. The system host processor maintains a long recordof the co-processor commands written into the command queue. Hardwaremonitors issue and completion of all of the commands in the queue, andrecords each event in a small memory. When the memory becomes full,software uses DMA to store the contents to main memory. After execution,the small memory traces are aggregated back into a single large trace bycorrelating their order with the long record maintained on the host.

Hardware/Software Support of Single-Stepping without a RestartableProcessor

Single-stepping a program is a debugging technique in which a piece ofcode is executed one instruction or “step” at a time, with theprogrammer inspecting the state of key variables or memory locationsafter each step. In principle, single-stepping requires the processor beable to pause and restart a program after each instruction. Adding theability to pause and restart a program to a high-performance processoradds even more complexity to a tightly-optimized design, which canreduce performance, increase area, or even introduce bugs.

In one embodiment the development system uses an innovative mechanismfor single-stepping of a program executed on a co-processor withoutrequiring the co-processor to support a program being paused andrestarted in mid-execution. Instead, the co-processor merely needs tosupport stopping on a specific instruction. The control software runningon the host processor then emulates running to an instruction n followedby single-stepping using one of three innovative approaches:

-   -   1. Save the co-processor state before executing each        co-processor command, then run the command to instruction n. To        single-step, restore the state and re-run to instruction n+1.        Since the state between processor commands does not include the        contents of the processor core, it is easy to save and restore.        Since the co-processor is very fast this behavior appears        identical to single-stepping to the user, but the hardware is        much simpler.    -   2. Run the command to instruction n, then save all of the        co-processor state including that of the processor core (using,        for instance, a scan-chain). Load the co-processor state into a        cycle-accurate simulator and single-step within the simulator.        Since the simulator is software, it is easy to extract any        desired information.    -   3. A hybrid approach in which the state before each processor        command is saved as in approach 1, single-stepping is performed        using approach 2, and execution can be resumed by restoring the        saved state as in approach 1.

Efficient Co-Processor Multi-Tasking

Efficient multitasking is a challenge in any processor-based system thatruns multiple applications. Embedded real-time systems are especiallychallenging since all tasks need to receive their input, process thisdata and present the corresponding output within allotted deadlines. Ofparticular interest are so-called “hard” real-time systems whichabsolutely have to complete their allocated work in time withoutexception or else suffer a fatal system failure, e.g. dropping videoinput content and/or failing to display video output on time.Conventional multitasking is typically based on “best effort”, i.e.execution of each task for some duration (time or event-determined) andswitching between these tasks based on priorities. Conventionalmultitasking does not meet the requirements of hard real-time systemsbecause there is no guarantee that deadlines will be met. The run-time(i.e., a library of functions made available to the software developer)introduces an innovative mechanism for coupling conventionalmultitasking running on a host processor with a more suitable (e.g.deadline-based) multitasking mechanism that switches between tasksrunning on the co-processor. This innovative coupling enables hardreal-time performance without requiring substantial changes to the hostoperating system.

The run-time includes an innovative multitasking mechanism that targetshardware containing a host CPU and specialized coprocessor (such as astream processor). The host CPU executes multiple tasks simultaneouslyusing conventional pre-emptive multi-tasking. Each task has a dedicatedco-processor command FIFO. When a task generates a co-processor command,it places it in its dedicated FIFO. The co-processor maintains a singlequeue of commands to be executed. When there is room in theco-processor's command queue, a special command dispatcher removes acommand from one task's dedicated FIFO and places it in theco-processor's command queue. This flow is illustrated in FIG. 24.

The command dispatcher may be implemented in software and/or hardware.An implementation in software may be an interrupt handler running on thehost CPU which responds to interrupts generated by the co-processor whenthe co-processor's command queue is either less than full or entirelyempty. When it receives an interrupt, the handler sends new command(s)to the co-processor's command queue. One implementation in hardwareinvolves actual hardware FIFOs used by tasks running on the host-CPU andhardwired dispatch logic to send commands from those FIFOs to theco-processor.

The command dispatcher may use one of several scheduling processes todecide from which task's FIFO to send a command to the co-processornext. One such process which is particularly suited to the demands ofhard real-time constraints found in DSP applications is deadlinescheduling. If deadline scheduling is used, it may either be implementedentirely in software, or using hardware support to monitor and trackdeadlines. The scheduling method may need to track and adjust todifferences between task execution estimates and actual executionestimates at run-time.

When the command dispatcher switches between a FIFO containing commandsfrom one task and a FIFO containing commands from another task, theco-processor may need to save state associated with the old task andrestore state associated with the new task. Saving and restoring ofstate can impose a high overhead, but this overhead may be reduced bythe addition of software or hardware support. One example of softwaresupport is having the programmer or compiler mark points in a task wherestate associated with that task is minimized and swapping would beinexpensive. One example of hardware support is using a multiple-bankon-chip memory which enables some state for the new task to be restoredwhile the old task is still running and enables the new task to beginrunning while state for the old task is being stored.

Operational Context

The embodiments described above may be implemented in a programmedgeneral-purpose or special-purpose computer system or in a network ofcomputer systems. Alternatively, the embodiments may be implemented in adevice that includes hardwired logic for carrying out theabove-described operations, or any combination of programmed processorsand hardwired logic.

FIG. 25 illustrates an exemplary a computer system 2500 upon whichembodiments of the invention may be implemented. Computer system 2500includes a bus 2502 or other communication mechanism for communicatinginformation, and a processing entity 2504 coupled with bus 2502 forprocessing information. The processing entity 2504 may include anynumber of general purpose and/or special purposes processors co-locatedwithin a single computing system or distributed over a network ofcomputing systems. Each integrated-circuit (IC) processor includedwithin the processing entity may include one or more classes of on-chipstorage as described above, as well as on-chip cache memory.

Computer system 2500 also includes a main memory 2506, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 2502for storing information and instructions to be executed by processingentity 2504 to carry out the above-described operations, includingexecuting the above described compiler program code and/orprogram-loading code (e.g., operating system code) that responds toinformation encoded during compiler execution into object files,executable files and the like. Main memory 2506 also may be used forstoring temporary variables or other intermediate information duringexecution of instructions to be executed by processing entity 2504.Computer system 2500 further includes a read only memory (ROM) 2508 orother static storage device coupled to bus 2502 for storing staticinformation and instructions for processing entity 2504. A storagedevice 2510, such as a magnetic disk or optical disk, is provided andcoupled to bus 2502 for storing information and instructions, such asthe functions and data structures described above.

Computer system 2500 may be coupled via bus 2502 to a display 2512, suchas a cathode ray tube (CRT) or flat-panel monitor, for displayinginformation to a computer user. An input device 2514, includingalphanumeric and other keys, is coupled to bus 2502 for communicatinginformation and command selections (e.g., the user-specified queriesdescribed above) to processing entity 2504. Another type of user inputdevice is cursor control 2516, such as a mouse, a trackball, or cursordirection keys for communicating direction information and commandselections to processing entity 2504 and for controlling cursor movementon display 2512.

Embodiments of the invention are related to the use of computer system2500 for implementing the techniques described herein. According to oneembodiment, those techniques are performed by computer system 2500 inresponse to processing entity 2504 executing one or more sequences ofone or more instructions contained in main memory 2506. Suchinstructions may be read into main memory 2506 from anothercomputer-readable medium, such as storage device 2510. Execution of thesequences of instructions contained in main memory 2506 causesprocessing entity 2504 to perform the process steps described herein. Inalternative embodiments, hard-wired circuitry may be used in place of orin combination with software instructions to implement the invention.Thus, embodiments of the invention are not limited to any specificcombination of hardware circuitry and software.

The term “computer-readable medium” as used herein refers to any mediumthat participates in providing instructions to processing entity 2504for execution. Such a medium may take many forms, including but notlimited to, non-volatile media, volatile media, and transmission media.Non-volatile media includes, for example, optical or magnetic disks,such as storage device 2510. Volatile media includes dynamic memory,such as main memory 2506. Transmission media includes coaxial cables,copper wire and fiber optics, including the wires that comprise bus2502. Transmission media can also take the form of acoustic or lightwaves, such as those generated during radio-wave and infra-red datacommunications.

Common forms of computer-readable media include, for example, a floppydisk, a flexible disk, hard disk, magnetic tape, or any other magneticmedium, a CD-ROM, any other optical medium, punchcards, papertape, anyother physical medium with patterns of holes, a RAM, a PROM, and EPROM,a FLASH-EPROM, any other memory chip or cartridge, a carrier wave asdescribed hereinafter, or any other medium from which a computer canread.

Various forms of computer readable media may be involved in carrying oneor more sequences of one or more instructions to processing entity 2504for execution. For example, the instructions may initially be carried ona magnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line or other signaling medium using a modem. A modem local tocomputer system 2500 can receive the data on the signaling medium anduse an infra-red transmitter to convert the data to an infra-red signal.An infra-red detector can receive the data carried in the infra-redsignal and appropriate circuitry can place the data on bus 2502. Bus2502 carries the data to main memory 2506, from which processing entity2504 retrieves and executes the instructions. The instructions receivedby main memory 2506 may optionally be stored on storage device 2510either before or after execution by processing entity 2504.

Computer system 2500 also includes a communication interface 2518coupled to bus 2502. Communication interface 2518 provides a two-waydata communication coupling to a network link 2520 that is connected toa local network 2522. For example, communication interface 2518 may bean integrated services digital network (ISDN) card or a modem to providea data communication connection to a corresponding type of telephoneline. As another example, communication interface 2518 may be a localarea network (LAN) card to provide a data communication connection to acompatible LAN. Wireless links may also be implemented. In any suchimplementation, communication interface 2518 sends and receiveselectrical, electromagnetic or optical signals that carry digital datastreams representing various types of information.

Network link 2520 typically provides data communication through one ormore networks to other data devices. For example, network link 2520 mayprovide a connection through local network 2522 to a host computer 2524or to data equipment operated by an Internet Service Provider (ISP)2526. ISP 2526 in turn provides data communication services through theworld wide packet data communication network now commonly referred to asthe “Internet” 2528. Local network 2522 and Internet 2528 both useelectrical, electromagnetic or optical signals that carry digital datastreams. The signals through the various networks and the signals onnetwork link 2520 and through communication interface 2518, which carrythe digital data to and from computer system 2500, are exemplary formsof carrier waves transporting the information.

Computer system 2500 can send messages and receive data, includingprogram code, through the network(s), network link 2520 andcommunication interface 2518. In the Internet example, a server 2530might transmit a requested code for an application program throughInternet 2528, ISP 2526, local network 2522 and communication interface2518.

The received code may be executed by processing entity 2504 as it isreceived, and/or stored in storage device 2510, or other non-volatilestorage for later execution. In this manner, computer system 2500 mayobtain application code in the form of a carrier wave.

While the invention has been described with reference to specificembodiments thereof, it will be evident that various modifications andchanges may be made thereto without departing from the broader spiritand scope of the invention. For example, features or aspects of any ofthe embodiments may be applied, at least where practicable, incombination with any other of the embodiments or in place of counterpartfeatures or aspects thereof. Accordingly, the specification and drawingsare to be regarded in an illustrative rather than a restrictive sense.

What is claimed is:
 1. A method of loading data into register files thatcorrespond to respective execution units within a processor, the methodcomprising: comparing a first set of parameters that specify a subset ofdata to be accessed from among a larger set of data within a firstmemory to a plurality of sets of conditions that correspond torespective patterns of data to identify one of the sets of conditionsthat satisfies the first set of parameters; converting the first set ofparameters to a second set of parameters in accordance with the patternof data corresponding to the one of the sets of conditions thatsatisfies the first set of parameters; generating, based on the secondset of parameters, a sequence of memory addresses that specify thesubset of data that is to be loaded from the first memory into theregister files that correspond to the respective execution units withinthe processor; and loading the subset of data from locations within thefirst memory specified by the sequence of memory addresses into theregister files that correspond to the respective execution units withinthe processor, wherein the first set of parameters that specify thesubset of data to be accessed within the first memory further comprises:a count value N that indicates a number of groups of data values to beloaded from the first memory relative to a predetermined startingaddress; a group parameter that indicates a number of data values to beread from sequentially-addressed locations within the first memory as agroup of data values; a stride parameter that indicates a value to beadded to the predetermined starting address of a first group of the datavalues to obtain a starting address of a second group of the datavalues; and wherein the method further comprises: retrieving 0 to Ngroups of data values from the first memory, wherein N corresponds tothe count value N, wherein each of the 0 to N groups are discontiguousin the first memory relative to each other, and wherein a final locationof a final group of the 0 to N groups in the first memory corresponds to((the count value N minus 1) multiplied by the stride parameter) plus(the group parameter minus 1); and contiguously storing the 0 to Ngroups of the data values as transformed data in a temporary buffer. 2.The method of claim 1, wherein loading the subset of data furthercomprises loading the transformed data from the temporary buffer intothe register files that correspond to the respective execution unitswithin the processor.
 3. The method of claim 1, wherein: the first setof parameters that specify the subset of data to be accessed within thefirst memory include an allocation parameter T that indicates adistribution of the 0 to N groups of the data values among the registerfiles that correspond to the respective execution units within theprocessor; and loading the subset of data comprises loading T of thegroups of the data values from the temporary buffer into each of theregister files.
 4. The method of claim 1, wherein the first set ofparameters that specify the subset of data to be accessed within thefirst memory include an allocation parameter that indicates adistribution of the subset of data among the register files thatcorrespond to the respective execution units within the processor. 5.The method of claim 4, wherein the step of loading the subset of datainto the register files that correspond to the respective executionunits within the processor comprises loading a first portion of thesubset of data into one of the register files and loading a secondportion of the subset of data into another of the register files,wherein each of the first and second portions of the subset of dataincludes a number of data values in accordance with the allocationparameter.
 6. The method of claim 1, wherein the first set of parametersthat specify the subset of data to be accessed within the first memoryfurther includes an offset value that indicates an offset between thestarting address of the first group of the data values and apredetermined address.
 7. The method of claim 6, wherein the first setof parameters that specify the subset of data to be accessed within thefirst memory further includes a pointer to a sequence of one or moreaddress values, and the sequence of one or more address values includesthe predetermined address.
 8. The method of claim 1, wherein the firstset of parameters that specify the subset of data to be accessed withinthe first memory further includes an allocation parameter that indicatesa number of execution units among which the first group of the datavalues is to be allocated.
 9. The method of claim 1, wherein the firstset of parameters that specify the subset of data to be accessed withinthe first memory further includes a count value that indicates a numberof group values to be loaded from the first memory relative to apredetermined starting address.
 10. A non-transitory computer-readablemedium having instructions stored thereon, the instructions comprising:instructions for comparing a first set of parameters that specify asubset of data to be accessed from among a larger set of data within afirst memory to a plurality of sets of conditions that correspond torespective patterns of data to identify one of the sets of conditionsthat satisfies the first set of parameters; instructions for convertingthe first set of parameters to a second set of parameters in accordancewith the pattern of data corresponding to the one of the sets ofconditions that satisfies the first set of parameters; instructions forgenerating, based on the second set of parameters, a sequence of memoryaddresses that specify the subset of data that is to be loaded from thefirst memory into register files that correspond to respective executionunits within a processor; and instructions for loading the subset ofdata from locations within the first memory specified by the sequence ofmemory addresses into the register files that correspond to therespective execution units within the processor, wherein the first setof parameters that specify the subset of data to be accessed within thefirst memory further comprises: a count value N that indicates a numberof groups of data values to be loaded from the first memory relative toa predetermined starting address; a group parameter that indicates anumber of data values to be read from sequentially-addressed locationswithin the first memory as a group of data values; a stride parameterthat indicates a value to be added to the predetermined starting addressof a first group of the data values to obtain a starting address of asecond group of the data values; and wherein the instructions furthercomprise: instructions for retrieving 0 to N groups of data values fromthe first memory, wherein N corresponds to the count value N, whereineach of the 0 to N groups are discontiguous in the first memory relativeto each other, and wherein a final location of a final group of the 0 toN groups in the first memory corresponds to ((the count value N minus 1)multiplied by the stride parameter) plus (the group parameter minus 1);and instructions for contiguously storing the 0 to N groups of the datavalues as transformed data in a temporary buffer.
 11. The non-transitorycomputer-readable medium of claim 10, wherein the instructions forloading the subset of data further comprises instructions for loadingthe transformed data from the temporary buffer into the register filesthat correspond to the respective execution units within the processor.12. The non-transitory computer-readable medium of claim 10, wherein:the first set of parameters that specify the subset of data to beaccessed within the first memory include an allocation parameter T thatindicates a distribution of the 0 to N groups of the data values amongthe register files that correspond to the respective execution unitswithin the processor; and the instructions for loading the subset ofdata comprises instructions for loading T of the groups of the datavalues from the temporary buffer into each of the register files. 13.The non-transitory computer-readable medium of claim 10, wherein thefirst set of parameters that specify the subset of data to be accessedwithin the first memory include an allocation parameter that indicates adistribution of the subset of data among the register files thatcorrespond to the respective execution units within the processor. 14.The non-transitory computer-readable medium of claim 13, wherein theinstructions for loading the subset of data into the register files thatcorrespond to the respective execution units within the processorcomprise instructions for loading a first portion of the subset of datainto one of the register files and loading a second portion of thesubset of data into another of the register files, wherein each of thefirst and second portions of the subset of data includes a number ofdata values in accordance with the allocation parameter.
 15. Thenon-transitory computer-readable medium of claim 10, wherein the firstset of parameters that specify the subset of data to be accessed withinthe first memory further include an offset value that indicates anoffset between the starting address of the first group of the datavalues and a predetermined address.
 16. The non-transitorycomputer-readable medium of claim 15, wherein the first set ofparameters that specify the subset of data to be accessed within thefirst memory include a pointer to a sequence of one or more addressvalues, and the sequence of one or more address values includes thepredetermined address.
 17. The non-transitory computer-readable mediumof claim 10, wherein the first set of parameters that specify the subsetof data to be accessed within the first memory further include anallocation parameter that indicates a number of execution units amongwhich the first group of the data values is to be allocated.
 18. Thenon-transitory computer-readable medium of claim 10, wherein the firstset of parameters that specify the subset of data to be accessed withinthe first memory further include a count value that indicates a numberof group values to be loaded from the first memory relative to apredetermined starting address.